Remote computing is the technique of abstracting some aspects of a computing system from a local host to a remote provider. Remote computing provides additional processing, capacity, storage, and security capabilities than would otherwise be available in local client devices Remote graphics computing is a particular implementation of remote computing that is rapidly gaining in popularity. Remote graphics computing is accomplished by performing the actual rendering and processing of the images constituting the graphical content displayed to the user in a remote device, such as a server. Once rendered, the image data is typically encoded (compressed) and transmitted as a stream of packets (bit stream) to the user's device (or client), where the encoded video stream is subsequently decoded and displayed.
User control input—typically responsive to the progression of images displayed—may be received from the user and directly transmitted back to the server, whereupon the input is recorded, and a new set of images may be processed, rendered, encoded, and then streamed back to the “client” device. This process may be performed continuously between the client device and the server, and often for substantially the entirety of the time the remote graphics computing session persists. Remote graphics computing allows users to view images while largely eliminating higher hardware requirements for the computing devices of users by abstracting the majority of the memory and graphical processing needs from the user's device and on to the server.
Unfortunately, a new host of challenges are also introduced by remote graphics processing over traditional, localized processing solutions. For example, additional steps are required to prepare the graphical output for display, such as compressing the video continuously in real time, decompressing the video at the user's client device, and continuously streaming data through a network connection. The challenge to perform these steps quickly and efficiently enough to not noticeably impact the user's experience while accommodating limitations due to the network infrastructure is considerable.
One potential drawback to remote processing is the introduction of additional latency experienced by the user. Latency is a measure of the responsiveness of the user's input to the server and back, and can be influenced by a host of variables, including how fast the server is running, the efficiency of the software and hardware running on the server, the network infrastructure, and the distance that the network has to travel to the user's device on the client side, and how fast the client device can decode the transmitted data. The further that the user's device is from the server for example, either in distance or network hops or network quality, the more latency that could be introduced due to lengthier transmission or routing times. Similarly, a user's device may be unable to decode the received data as quickly as the data is received, and lengthy data queues may result. Long latencies, often referred to as “lag,” are undesirable, particularly during competitive games, time-sensitive interactions in games, and/or critical portions during videos, and can have a significant negative effect on user experience.
Unfortunately, many traditional remote processing solutions do not account for network or client attributed lag, considering them to be unavoidable or inevitable. At best, remote processing system operators may attempt to manage latencies by provisioning additional resources at the server side to expedite processing. However, this can increase capital expenditure costs for the remote processing system operators, potentially prohibitively, and is neither an efficient nor effective solution since the client device can still be overloaded. Still other remote processing system operators may attempt to avoid latencies altogether by limiting the applications offered, either by limiting the complexity of the images being rendered and the size and/or frequency of the data transmissions, providing only graphical content with asynchronous and/or mechanisms unrelated to timing. Unfortunately, this also restricts user choice in graphical content, and may prevent users from accessing a fuller library of graphical entertainment.